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DETAILED ACTION 

Remarks 

1 . This communication is in response to the amendment filed on May 1 0, 201 0. 

2. Claims 2, 8-9, 13-15, 17, 21-22, 26-37, 39-40 and 42-53 have been cancelled 
and claim 55-56 have been added. Therefore, claims 1, 3-7, 10-12, 16, 18-20, 23-25, 
38, 41 and 54-56 are pending in the application for examination. 

Claim Rejections - 35 USC §112 

3. The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

Claims 1, 16, 38 and 41 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. Claims 1 , 1 6, 38 and 41 are vague and 
indefinite because the phrase "prior application with focus" has not been clearly defined 
in the claims, such as what prior application, what is meant by "focus" and how the 
application is in focus. Although the claims are interpreted in light of the specification, 
limitations from the specification are not read into the claims. See In re Van Geuns, 988 
F.2d 1181, 26 USPQ2d 1057 (Fed. Cir.1993). Therefore, one of ordinary skill in the art 
would not be able to ascertain the metes and bounds of the claimed invention. 

The Examiner has given the phrase "prior application with focus" its broadest 
reasonable interpretation. For examination purposes, all claim interpretation is 
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predicated upon the broadest reasonable interpretation of the claim terms which would 
be fairly conveyed to one of ordinary skill in the pertinent art. In this case, Examiner 
interpret "prior application with focus" as an application that a user was actively working 
before the current active application, changing windows of applications using mouse 
action or keyboard action. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1 , 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 

6. Claims 1, 38, 55 and 56 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Schumacher et al. (US Patent No. 6,631,345, ' Schumacher ', 
hereafter, previously provided) in view of Chang et al. (US Patent No. 6,968,509, 
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' Chang ', hereafter) and further in view of Gray et al. (US patent Application No. 
2005/0060719, ' Gray ', hereafter). 

Regarding claim 1, Schumacher teaches a computer implemented method for a 
capture processor executing on a computer to determine an event associated with an 
application (A method, system, and computer program product for emulating a 
sequence of events resulting from user interaction with an applet in which the storing 
and retrieval of queued event objects is facilitated through the use of an index to a 
component vector, Schumacher , Abstract and Col 2, lines 14-17), comprising: 

receiving, with the capture processor, a plurality of new keystrokes associated 
with the new application; processing, with the capture processor, each new keystroke to 
determine an associated action in the new application the plurality of new keystrokes 
forming a plurality of associated actions ( Schumacher, FIG. 1, includes an Automator. 
The applet event recorder 100 with Automator 102, when invoked, provides the user 
with a graphical user interface for associating an applet 108 that is to be monitored and 
for selecting user interaction extent types for monitoring. Automator 102 then places 
Automator listener objects 104 on components of applet 108. Automator listeners 104 
are suitably configured to store detected events to an Automator queue 106 as queued 
event objects. If, for example, it is desirable to record only keyboard entries such as by 
way of AWT Key Event object, Automator 102 provides a simple and friendly user 
interface enabling one to select one or more of multiple types of events that may be 
recorded, Schumacher. Col 4, lines 41-65); 
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selectively indexing the complete event responsive to determining that the 
complete event occurred {capturing and storing keyboard entries and selectively 
queued them, Schumacher, Col 4, line 60 - Col 5, line 12 and Col 2, lines 14-35). 

Schumachar does not teach that 

receiving, with the capture processor, a plurality of keystrokes associated with a 
prior application with focus monitored by the capture processor; 

determining, with the capture processor, that the focus has changed from the 
prior application monitored by the capture processor to a new application monitored by 
the capture processor; 

analyzing, with the capture processor, the plurality of associated actions to 
determine whether a complete event has occurred in the new application. 

However, Chang teaches that 

receiving, with the capture processor, a plurality of keystrokes associated with a 
prior application with focus monitored by the capture processor {CPU receives user 
activities such as keystrokes on the keyboard. CPU executing dynamic-link library that 
access window messages from the window of the application with focus to the operating 
system of the computer, Chang , Col 5, lines 4-10, Col 6, lines 4-10 and Figs. 2-3. The 
present invention records user-driven events within an application of a computer by 
detecting changes in a focus of the application while monitoring for window messages 
from the application window to the operating system of the computer. The focus of the 
application includes where a mouse pointer is located on a graphical user interface of 
the application, such as a focus on a particular button of a toolbar that causes the 
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button to have a raised appearance. The window message specifies user activity such 
as a click of a mouse button and pressing of keys on the keyboard. The focus that 
results from each change of focus is logged. When a window message occurs, a focus 
that has been logged and the user activity of the window message can be recorded to 
specify the user-driven event, Chang, Col 1, lines 54-67); 

determining, with the capture processor, that the focus has changed from the 
prior application monitored by the capture processor to a new application monitored by 
the capture processor (change in focus, Chang. Col 6, lines 19-37, Fig. 4 and Chang. 
Col 1, lines 54-67); 

analyzing, with the capture processor, the plurality of associated actions to 
determine whether a complete event has occurred in the new application (Each 
character that is typed may be placed on the same line of the textual description as 
shown in the screenshots until the typed key is an <Enter> key or unless there is a 
control key such as <Backspace> or <Ctrl>+"A". After each keystroke is recorded, 
operational flow returns to signal operation (i.e., awaiting for the new activity after 
completing the prior activities), Chang . Col 5, lines 15-26 and Figs. 2, 7-8). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar and Chang before 
him/her to modify Schumachar with the teaching of Chang 's recording of user-driven 
events within a computer application. One would have been motivated to do so for the 
benefit of recording user-driven events including a memory and a processor. The 
processor executes the application and the graphical user interface provided to the user 



Application/Control Number: 10/814,773 Page 7 

Art Unit: 2168 

through a display device. The processor detects the changes of focus of the application 
and records a focus resulting from each change as an entry to memory. The processor 
also monitors for a window message specifying user activity while detecting the 
changes of focus. When a window message occurs, the processor specifies the user- 
driven event by recording the user activity of the message and also recording a focus of 
an entry stored in the memory as taught by Chang (Col 2, lines 4-15). 
Schumachar and Chang explicitly do not teach that 

resetting, with the capture processor, the keystrokes captured from the prior 
application by clearing the captured keystrokes responsive to determining that the focus 
has changed; 

However, Gray teaches that 

resetting, with the capture processor, the keystrokes captured from the prior 
application by clearing the captured keystrokes responsive to determining that the focus 
has changed ( Gray . [0042] and Fig. 3 illustrates that the "New" command resets the 
memory of event engines and initializes states for a new recording (i.e., reset the 
capture processor and change the focus); 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar , Chang and Gray 
before him/her to further modify Schumachar with the teaching of Gray 's capturing and 
processing user events on a computer system for recording and playback. One would 
have been motivated to do so for the benefit of capturing and processing user events 
that are associated with screen objects that appear on a computer display device. User 
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events may be captured and recorded so that the user events may be reproduced either 
at the user's computer or at another computer as taught by Gray ([0006]). 

Regarding claim 38, although claim 38 is directed to a computer-readable 
storage medium, it is similar in scope to claim 1 . It would be obvious to implement the 
method of claim 1 on a computer-readable storage medium; the method of claim 1 
would inherently involve the need for the method to be implemented on a computer- 
readable storage medium. The method steps of claims claim 1 substantially 
encompass the computer-readable storage medium recited in claim 38 therefore; claim 
38 is rejected for at least the same reason as claim 1 above. 

Regarding claim 55, Schumachar and Chang do not teach that the resetting the 
keystrokes captured from the prior application comprises saving the keystrokes before 
clearing the keystrokes. 

However, Gray teaches that the resetting the keystrokes captured from the prior 
application comprises saving the keystrokes before clearing the keystrokes ( Gray. 
[0042] and [0006-0008]). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar , Chang and Gray 
before him/her to further modify Schumachar with the teaching of Gray 's capturing and 
processing user events on a computer system for recording and playback. One would 
have been motivated to do so for the benefit of capturing and processing user events 
that are associated with screen objects that appear on a computer display device. User 
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events may be captured and recorded so that the user events may be reproduced either 
at the user's computer or at another computer as taught by Gray ([0006]). 

Regarding claim 56, Schumachar and Gray do not teach that an application has 
focus if a user of the computer is currently using the application to input text to the 
computer. 

However, Chang teaches that an application has focus if a user of the computer 
is currently using the application to input text to the computer ( Chang, Col 4, lines 4-12, 
47-53, Col 5, lines 15-27 and Fig. 3-5). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar , Gray and Chang 
before him/her to modify Schumachar with the teaching of Chang 's recording of user- 
driven events within a computer application. One would have been motivated to do so 
for the benefit of recording user-driven events including a memory and a processor. The 
processor executes the application and the graphical user interface provided to the user 
through a display device. The processor detects the changes of focus of the application 
and records a focus resulting from each change as an entry to memory. The processor 
also monitors for a window message specifying user activity while detecting the 
changes of focus. When a window message occurs, the processor specifies the user- 
driven event by recording the user activity of the message and also recording a focus of 
an entry stored in the memory as taught bv Chang (Col 2, lines 4-15). 
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7. Claims 3-5, 10 and 54 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Schumacher et al. (US Patent No. 6,631,345, ' Schumacher ', 
hereafter, previously provided) in view of Chang et al. (US Patent No. 6,968,509, 
' Chang ', hereafter) further in view of Gray et al. (US patent Application No. 
2005/0060719, ' Gray ', hereafter, previously provided) and further in view of Weber et al. 
(US Patent Number 5,305,205, 'Weber', hereafter, previously provided). 

Regarding claim 3, Schumachar , Chang and Gray do not teach that wherein the 
analyzing determines that a complete event has occurred responsive to the plurality of 
associated actions indicating that a complete word has been entered into the 
application. 

However, Weber teaches that wherein the analyzing determines that a complete 
event has occurred responsive to the plurality of associated actions indicating that a 
complete word has been entered into the application ( Weber . Col 5, lines 5-52). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar . Chang . Gray and 
Weber before him/her to further modify Schumachar with the teaching of Weber 's 
computer-assisted transcription apparatus. One would have been motivated to do so 
for the benefit of having an improved word processing and transcription system for 
generating a display list of words and phrases following an initial entry of some or all 
letters of text by a user for enabling the user to selectively enter a word or phrase from 
the list into a word processor as taught by Weber (Col 2, lines 8-13). 
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Regarding claim 4, Schumachar , Chang and Gray do not teach that wherein the 
analysis determines that a complete word has been entered responsive to the plurality 
of associated actions indicating that a space or a punctuation symbol has been entered. 

However, Weber teaches that wherein the analysis determines that a complete 
word has been entered responsive to the plurality of associated actions indicating that a 
space or a punctuation symbol has been entered ( Weber , Col 5, lines 5-52). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar , Chang , Gray and 
Weber before him/her to further modify Schumachar with the teaching of Weber 's 
computer-assisted transcription apparatus. One would have been motivated to do so 
for the benefit of having an improved word processing and transcription system for 
generating a display list of words and phrases following an initial entry of some or all 
letters of text by a user for enabling the user to selectively enter a word or phrase from 
the list into a word processor as taught by Weber (Col 2, lines 8-13). 

Regarding claim 5, Schumachar , Chang and Gray do not teach that wherein the 
analyzing determines that a complete event has occurred responsive to the plurality of 
associated actions indicating that a predetermined number of characters have been 
typed into the application. 

However, Weber teaches that wherein the analyzing determines that a complete 
event has occurred responsive to the plurality of associated actions indicating that a 
predetermined number of characters have been typed into the application ( Weber , Col 
5, lines 5-52). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar , Chang , Gray and 
Weber before him/her to further modify Schumachar with the teaching of Weber 's 
computer-assisted transcription apparatus. One would have been motivated to do so 
for the benefit of having an improved word processing and transcription system for 
generating a display list of words and phrases following an initial entry of some or all 
letters of text by a user for enabling the user to selectively enter a word or phrase from 
the list into a word processor as taught by Weber (Col 2, lines 8-1 3). 

Regarding claim 10, Schumachar , Chang and Gray do not teach that an 
associated action comprises one of adding a character to a word, deleting a character 
from a word, inserting a character, overwriting a character, deleting a word, deleting a 
paragraph, selecting an item, and repositioning the cursor. 

However, Weber teaches that an associated action comprises one of adding a 
character to a word, deleting a character from a word, inserting a character, overwriting 
a character, deleting a word, deleting a paragraph, selecting an item, and repositioning 
the cursor ( Weber . Col 3, lines 40-42, 55-59 and Col 5, lines 5-14). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar , Chang , Gray and 
Weber before him/her to further modify Schumachar with the teaching of Weber 's 
computer-assisted transcription apparatus. One would have been motivated to do so 
for the benefit of having an improved word processing and transcription system for 
generating a display list of words and phrases following an initial entry of some or all 
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letters of text by a user for enabling the user to selectively enter a word or phrase from 
the list into a word processor as taught by Weber (Col 2, lines 8-13). 

Regarding claim 54, Schumachar , Chang and Gray do not teach that the 
analyzing determines that a complete event has occurred responsive to the plurality of 
associated actions indicating that a predetermined number of words have been typed 
into the application. 

However, Weber teaches that the analyzing determines that a complete event 
has occurred responsive to the plurality of associated actions indicating that a 
predetermined number of words have been typed into the application (Weber, Col 5, 
lines 5-52). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar . Chang . Gray and 
Weber before him/her to further modify Schumachar with the teaching of Weber 's 
computer-assisted transcription apparatus. One would have been motivated to do so 
for the benefit of having an improved word processing and transcription system for 
generating a display list of words and phrases following an initial entry of some or all 
letters of text by a user for enabling the user to selectively enter a word or phrase from 
the list into a word processor as taught by Weber (Col 2, lines 8-13). 

8. Claims 6 and 7 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Schumacher et al. (US Patent No. 6,631,345, ' Schumacher ', hereafter, previously 
provided) in view of Chang et al. (US Patent No. 6,968,509, ' Chang ', hereafter) and 
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further in view of Gray et al. (US patent Application No. 2005/0060719, ' Gray ', 
hereafter) and further in view of Yee et al. (US Patent No. 6,380,924, ' Yee ', hereafter, 
previously provided). 

Regarding claim 6, Schumachar , Chang and Gray do not teach that updating, 
with the capture processor, a capture state after each keystroke is processed. 

However, Yee teaches that updating, with the capture processor, a capture state 
after each keystroke is processed (in the MCR environment, one can record all the 
keystrokes and mouse actions to play the event of an action such as open file, save file 
etc. in a word processing software, Yee, Col 4, lines 39-43). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar . Chang . Gray and 
Yee before him/her to further modify Schumachar as modified with the teaching of Yee 's 
mouse/keyboard capture recorder (MCR) apparatus and methodology. One would be 
motivated to do so for the benefit of recording all the keystrokes and mouse actions 
needed to take the user to the data entry point of any application as taught by Yee ( Yee . 
Col 4, lines 39-43). 

Regarding claim 7, Schumachar . Chang and Gray do not teach that updating, 
with the capture processor, a current user state based at least in part on the event. 

However, Yee teaches that updating, with the capture processor, a current user 
state based at least in part on the event (in the MCR environment, one can record all 
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the keystrokes and mouse actions to play the event of an action such as open file, save 
file etc. in a word processing software, Yee . Col 4, lines 39-43). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar , Chang , Gray and 
Yee before him/her to further modify Schumachar as modified with the teaching of Yee 's 
mouse/keyboard capture recorder (MCR) apparatus and methodology. One would be 
motivated to do so for the benefit of recording all the keystrokes and mouse actions 
needed to take the user to the data entry point of any application as taught by Yee ( Yee , 
Col 4, lines 39-43). 

9. Claims 11 and 12 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Schumacher et al. (US Patent No. 6,631,345, ' Schumacher ', hereafter, previously 
provided) in view of Chang et al. (US Patent No. 6,968,509, ' Chang ', hereafter) and 
further in view of Gray et al. (US patent Application No. 2005/0060719, ' Gray ', 
hereafter) and further in view of Tervo et al. (US Patent Number 6,907,577, provided by 
the applicant's IDS). 

Regarding claim 1 1 , Schumachar , Chang and Gray do not teach that the 
associated action is determined based at least in part by matching a keystroke to a 
keystroke table and wherein the keystroke table is associated with the application and 
wherein different applications are associated with different keystroke tables. 
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However, Tervo teaches that the associated action is determined based at least 
in part by matching a keystroke to a keystroke table and wherein the keystroke table is 
associated with the application and wherein different applications are associated with 
different keystroke tables ( Tervo discloses a keystroke database having a number of 
keystrokes associated with a number of screens. Each keystroke may perform a 
different function or be associated with a different field based upon which screen of the 
possible screens that is currently active. Tervo also discloses a computer program 
determines that the ALT key in combination with another key has been depressed on a 
keyboard. It then identifies the current screen that is active. It then accesses a 
keystroke database to determine a field or function associated with the key depressed 
and the screen that is currently active. Then it activates the field locator module when 
the keystroke database indicates a field is desired for the key depressed, or it activates 
a function when the keystroke database indicates a function is desired for the key 
depressed, Tervo, Abstract and Col 2, lines 3-28. Tervo. Col 5, lines 30-46 and Fig. 6 
illustrates that when a user presses ALT C - ALT Q the processor captures these 
actions, matches with the Keystroke database as in Fig. 2 and produce a blank client 
field and calls the client building function as an event. For similar actions and events 
please see Tervo, Col 5, line 47- Col 6, line 12 and Fig. 7-8). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Schumachar . Chang . Gray and 
Tervo before him/her to modify Schumachar with the teaching of Tervo 's system and 
method for accessing screen fields, functions and programs using a simple single key 
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stroke. One would have been motivated to do so for the benefit of having a keystroke 
database where the keystrokes are associated with the applications active screen and 
windows and each keystroke may perform a different function or be associated with a 
different field based upon which screen of the possible screens that is currently active 
as taught by Tervo ( Tervo , Col 2, lines 1-15). 

Regarding claim 12, Schumachar , Chang and Gray do not teach that the 
associated action is determined based at least in part by matching a keystroke to a 
generic keystroke table common to a plurality of applications. 

However, Tervo teaches that the associated action is determined based at least in 
part by matching a keystroke to a generic keystroke table common to a plurality of 
applications ( Tervo discloses a keystroke database having a number of keystrokes 
associated with a number of screens. Each keystroke may perform a different function 
or be associated with a different field based upon which screen of the possible screens 
that is currently active. Tervo also discloses a computer program determines that the 
ALT key in combination with another key has been depressed on a keyboard. It then 
identifies the current screen that is active. It then accesses a keystroke database to 
determine a field or function associated with the key depressed and the screen that is 
currently active. Then it activates the field locator module when the keystroke database 
indicates a field is desired for the key depressed, or it activates a function when the 
keystroke database indicates a function is desired for the key depressed, Tervo, 
Abstract and Col 2, lines 3-28. Tervo , Col 5, lines 30-46 and Fig. 6 illustrates that when 
a user presses ALT C - ALT Q the processor captures these actions, matches with the 
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Keystroke database as in Fig. 2 and produce a blank client field and calls the client 
building function as an event. For similar actions and events please see Tervo, Col 5, 
line 47 - Col 6, line 12 and Fig. 7-8). 

Therefore, it would have been obvious to one of ordinary skill in the art at the time 
the invention was made having the teachings of Schumachar , Chang , Gray and Tervo 
before him/her to modify Schumachar with the teaching of Tervo 's system and method 
for accessing screen fields, functions and programs using a simple single key stroke. 
One would have been motivated to do so for the benefit of having a keystroke database 
where the keystrokes are associated with the applications active screen and windows 
and each keystroke may perform a different function or be associated with a different 
field based upon which screen of the possible screens that is currently active as taught 
by Tervo ( Tervo , Col 2, lines 1-15). 

10. Claims 16, 18 and 41 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Gray et al. (US patent Application No. 2005/0060719, ' Gray ', hereafter, previously 
provided) in view of Chang et al. (US Patent No. 6,968,509, ' Chang ', hereafter). 

Regarding claim 16, Gray teaches that a computer-implemented method for a 
capture processor executing on a computer to determine and selectively index an event 
associated with an application (a user interface supports a plurality of commands 
through a window that is displayed at the user's computer. The command types include 
recording user events, saving a file representing the user events, loading the file, 
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playing back the file to reproduce the user events, viewing the file, and adding notes to 
the file, Gray, [0010]), comprising: 

receiving, with the capture processor, a plurality of new display calls associated 
with the new application (event engine receives and evaluates "record" command from 
user interface, Gray , [0043], Fig. 4, 405. User events may be captured and recorded so 
that the user events may be reproduced either at the user's computer or at another 
computer. An event engine is instructed, through a user interface, to capture and to 
process a user event that is applied to a screen object, Gray. Abstract); 

processing, with the capture processor, the plurality of new display calls to 
determine a display produced by the new application ( Gray. [0010] and [0043]); 

resetting, with the capture processor, the display calls captured from the prior 
application by clearing the captured display calls responsive to determining that the 
focus has changed ( Gray , [0042] and Fig. 3 illustrates that the "New" command resets 
the memory of event engines and initializes states for a new recording (i.e., reset the 
capture processor and change the focus); 

Gray does not teach that 



receiving, with the capture processor, a plurality of display calls associated with a 
prior application with focus monitored by the capture processor; 

determining, with the capture processor, that the focus has changed from the 
prior application monitored by the capture processor to a new application monitored by 
the capture processor; 
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analyzing, with the capture processor, the display produced by the new 
application to determine whether a complete event has occurred in the new application; 

determining, with the capture processor, an importance of the complete event; 
and selectively indexing, with the capture processor, the complete event responsive to 
the importance of the complete event. 

However, Chang teaches that 



receiving, with the capture processor, a plurality of display calls associated with a 
prior application with focus monitored by the capture processor {CPU receives user 
activities such as keystrokes on the keyboard. CPU executing dynamic-link library that 
access window messages from the window of the application with focus to the operating 
system of the computer, Chang , Col 5, lines 4-10, Col 6, lines 4-10 and Figs. 2-3); 

determining, with the capture processor, that the focus has changed from the 
prior application monitored by the capture processor to a new application monitored by 
the capture processor (change in focus, Chang. Col 6, lines 19-37 and Fig. 4); 

analyzing, with the capture processor, the display produced by the new 
application to determine whether a complete event has occurred in the new application 
(within the currently focused application, each character that is typed may be placed on 
the same line of the textual description as shown in the screenshots until the typed key 
is an <Enter> key or unless there is a control key such as <Backspace> or <Ctrl>+"A". 
After each keystroke is recorded, operational flow returns to signal operation (i.e., 
awaiting for the new activity after completing the prior activities), Chang, Col 5, lines 15- 
26 and Figs. 2, 7-8). 
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determining, with the capture processor, an importance of the complete event; 
and selectively indexing, with the capture processor, the complete event responsive to 
the importance of the complete event (saving/storing user activities on a stack, Chang, 
Col 5, lines 27-37). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Gray and Chang before him/her to 
modify Gray with the teaching of Chang 's recording of user-driven events within a 
computer application. One would have been motivated to do so for the benefit of 
recording user-driven events including a memory and a processor. The processor 
executes the application and the graphical user interface provided to the user through a 
display device. The processor detects the changes of focus of the application and 
records a focus resulting from each change as an entry to memory. The processor also 
monitors for a window message specifying user activity while detecting the changes of 
focus. When a window message occurs, the processor specifies the user-driven event 
by recording the user activity of the message and also recording a focus of an entry 
stored in the memory as taught by Chang (Col 2, lines 4-15). 

Regarding claim 18, Gray as modified teaches that wherein the analyzing 
determines that a complete event has occurred responsive to the display indicating that 
a complete word has been entered into the application ( Gray, FIG. 2 illustrates that a 
user is manipulating a mouse and a keyboard to generate user events that are 
associated with an application 205. In the embodiment, application 205 is a software 
program, including a database manager, spreadsheet, communications package, 
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graphics package, word processor, and web browser. The user is operating on desktop 
201. For example, the user may click or double-click on a screen object (associated with 
application 205) or may enter text into a window corresponding to application 205. The 
user may activate the capturing and processing of user events by entering commands 
through a user interface 207 such as entering a record command, Gray, [0036]). 

Regarding claim 41, although claim 41 is directed to a computer-readable 
medium, it is similar in scope to claim 16. It would be obvious to implement the method 
of claim 16 on a computer-readable medium; the method of claim 16 would inherently 
involve the need for the method to be implemented on a computer-readable medium. 
The method steps of claims claim 16 substantially encompass the computer-readable 
medium recited in claim 41 therefore; claim 41 is rejected for at least the same reason 
as claim 16 above. 

1 1 . Claims 1 9, 20 and 23-25 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over Gray et al. (US patent Application No. 2005/0060719, ' Gray ', 
hereafter, previously provided) in view of Chang et al. (US Patent No. 6,968,509, 
' Chang ', hereafter) and further in view of Jade et al. (US Pub Number 2003/0001854, 
provided by the applicant's IDS). 

Regarding claim 19, Gray and Chang do not teach that updating, with the 
capture processor, a capture state after each display call is processed. 
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However, Jade teaches that updating, with the capture processor, a capture state 
after each display call is processed (the patches allow for the capture of the various 
graphics primitives (display calls) and associated attributes of the primitives that are 
drawn to the user interface, Jade , [001 1], lines 15-17). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Gray , Chang and Jade before 
him/her to further modify Gray with the teaching of Jade's capturing graphics primitives 
associated with any display object rendered to a graphical user interface. One would 
have been motivated to do so for the benefit of capturing the one or more graphics 
primitives associated with an application as it is in execution and applying them directly 
to any controls, buttons, windows and/or any other display objects that can be invoked 
by an application with respect to the operating system as taught by Jade ([0010]). 

Regarding claim 20, Gray and Chang do not teaches that updating, with the 
capture processor, a current user state based at least in part on the event 

However, Jade teaches that updating, with the capture processor, a current user 
state based at least in part on the event (a "calling process" is the process that utilizes 
the invention to capture the one or more graphics primitives of a display object (display 
elements) that can be invoked by the various application programs on the computer, 
Jade . [0023]). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Gray , Chang and Jade before 
him/her to further modify Gray with the teaching of Jade's capturing graphics primitives 
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associated with any display object rendered to a graphical user interface. One would 
have been motivated to do so for the benefit of capturing the one or more graphics 
primitives associated with an application as it is in execution and applying them directly 
to any controls, buttons, windows and/or any other display objects that can be invoked 
by an application with respect to the operating system as taught by Jade ([0010]). 

Regarding claim 23, Gray and Chang do not teaches that the display is 
determined at least in part by using an array of a current state of the display and 
updating the array with the display call, and wherein the analyzing comprises analyzing 
the array to determine whether a complete event has occurred. 

However, Jade teaches that the display is determined at least in part by using an 
array of a current state of the display and updating the array with the display call, and 
wherein the analyzing comprises analyzing the array to determine whether a complete 
event has occurred (This descriptive information can include parameters such as the 
type of display object (dialog box, menu, window, etc.) and its current state 
(active/inactive). Context information also includes system information such as the API 
calls and/or function calls made by the target application to render the display object to 
a user interface, the object handle or resource ID, the specific location of files called 
during execution of the display object, and any other information that provides a general 
context for the text that is displayed to the user interface screen 191 during the 
execution of the target process or application, Jade . [0026]. In addition it is also well 
known in the art that display is an array of the pixels and the current state of the display 
would be determined by the array of the pixels). 
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Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Gray , Chang and Jade before 
him/her to further modify Gray with the teaching of Jade's capturing graphics primitives 
associated with any display object rendered to a graphical user interface. One would 
have been motivated to do so for the benefit of capturing the one or more graphics 
primitives associated with an application as it is in execution and applying them directly 
to any controls, buttons, windows and/or any other display objects that can be invoked 
by an application with respect to the operating system as taught by Jade ([0010]). 

Regarding claim 24, Gray and Chang do not teaches that 

However, Jade teaches that the display is determined at least in part by 
constructing display items based at least in part on display positions of the display calls 
(a display object is invalidated each time a user resizes the display object or moves it to 
a different position within the user interface, Jade , [0039]). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Gray , Chang and Jade before 
him/her to further modify Gray with the teaching of Jade's capturing graphics primitives 
associated with any display object rendered to a graphical user interface. One would 
have been motivated to do so for the benefit of capturing the one or more graphics 
primitives associated with an application as it is in execution and applying them directly 
to any controls, buttons, windows and/or any other display objects that can be invoked 
by an application with respect to the operating system as taught by Jade ([0010]). 

Regarding claim 25, Gray and Chang do not teaches that 
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However, Jade teaches that processing the plurality of display calls to determine 
a display comprises analyzing one or more of the x,y coordinates, lengths, and relative 
positions of a plurality of items written to the display using display calls ( Jade teaches 
the graphics primitives include drawing elements (display items) such as text characters 
or strings, lines, arcs, polygons, etc., and have associated attributes that define its 
visual appearance such as font size, line length, and arc length, Jade , [0023], lines 7- 

1 1 . In addition, the x,y coordinates and relative positions are well known in the art 
especially in graphical user interface (GUI)). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made having the teachings of Gray , Chang and Jade before 
him/her to further modify Gray with the teaching of Jade's capturing graphics primitives 
associated with any display object rendered to a graphical user interface. One would 
have been motivated to do so for the benefit of capturing the one or more graphics 
primitives associated with an application as it is in execution and applying them directly 
to any controls, buttons, windows and/or any other display objects that can be invoked 
by an application with respect to the operating system as taught by Jade ([0010]). 

Response to Arguments 

12. Applicant's arguments with respect to claims 1 , 3-7, 1 0-1 2,16,1 8-20, 23-25, 38, 
41 and 54-56 have been considered but are moot in view of the new ground(s) of 
rejection in view of Chang et al. (US Patent No. 6,968,509). 



Application/Control Number: 10/814,773 Page 27 

Art Unit: 2168 

1 3. In response to applicant's argument on pages 9 and 1 0 that Gray does not teach 
that "resetting, with the capture processor, the keystrokes captured from the prior 
application by clearing the captured keystrokes responsive to determining that the focus 
has changed", acknowledged but deemed not to be persuasive. 

Gray , [0006-0008] illustrates that the event engines captures and process user 
events (i.e., keystrokes or mouse actions). The user events may be captured and 
recorded so that the user events may be reproduced either at the user's computer or at 
another computer (i.e., saving the user events) and the event engine interacts with one 
or more application programming interfaces (APIs) that may be supported by the 
applications being monitored (i.e., the application in focus). Gray , [0042] and Fig. 3 
illustrates that the "New" command resets the memory of event engines and initializes 
states for a new recording (i.e., reset the capture processor and change the focus). So 
when the "New" command executes, event engines resets (i.e., clears their memory) 
and returns to the step of acquiring new user events with a change of focus. Thus, 
Gray discloses the above argued limitation of claim 1 . 

14. In response to applicant's argument on pages 9 and 10 that the references fail to 
show certain features of applicant's invention, it is noted that the features upon which 
applicant relies (i.e., highlight any specific actions that are performed responsive to a 
focus change) are not recited in the rejected claim(s). Although the claims are 
interpreted in light of the specification, limitations from the specification are not read into 
the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). 
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Conclusion 

1 5. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 

§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

16. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to HASANUL MOBIN whose telephone number is 
(571)270-1289. The examiner can normally be reached on Monday Thru Friday 5:30 to 
1 :00 and Saturday. If attempts to reach the examiner by telephone are unsuccessful, 
the examiner's supervisor, Tim Vo can be reached on 571-272-3642. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

/Tim T. Vo/ 

Supervisory Patent Examiner, Art 
Unit 2168 

/H. M./ 

Examiner, Art Unit 2168 



